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Abstract 

We study evolutionary game dynamics in a well-mixed populations of finite size, N. A 
well-mixed population means that any two individuals are equally likely to interact. In 
particular we consider the average abundances of two strategies, A and B, under mutation 
and selection. The game dynamical interaction between the two strategies is given by the 
2x2 payoff matrix ( ^ ^ ) ■ It has previously been shown that A is more abundant than 
B, if a{N — 2) + ^A*" > cN + d{N — 2). This result has been derived for particular 
stochastic processes that operate either in the limit of asymptotically small mutation rates 
or in the limit of weak selection. Here we show that this result holds in fact for a wide 
class of stochastic birth-death processes for arbitrary mutation rate and for any intensity of 
selection. 
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1 Introduction 



Evolutionary dynamics describe how successful strategies spread in a population 
through genetic reproduction or cultural imitation. In mutation-selection processes 
where the fitness of each individual is constant, the competition between two types 
A and B is st raightforward. If both fitness values are identical, then we have neu- 



tral evolution (|KimuraLll968|) and the average abundances of A and B are the same. 
(Throughout the paper we assume symmetric mutation rates: the mutation proba- 
bility from A to i? is the same as from B to A.) If A is fitter than B, then A is more 
abundant than B in the mutation selection equilibrium. Thus, for constant selection 
the comparison of the abundances of the two types is trivial. 

For frequency dependent selection, where fitness depends on the types and abun- 
dances of others, the situation is more complex. One way to model these systems 
are evolutionary games, where the reproductive success of individuals depends on 
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their payoff derived from intera c tion with others (IMavnard Smith and Pried 1 1973 



Mavnard Smith' Il982l: IW eibuU'. '1995; 'Samuelson, 'l997'; 'Hofbauer and Sigmund, 



1998|;iFudenberg and Tirole, 1998; Hofbauer and Sig mund,.2003.;,Nowak and SigmundL 



2004l;Ea 



Szabo and path, 2007; Sandholm. 2007) . In the simplest case, interactions 



are described by a 2 x 2 payoff matrix. 
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Here, an A individual obtains a from other A individuals, but b from B individuals. 
Similarly, B obtains c from A, and d from other B individuals. 

For a > c and b > d, strategy A dominates strategy B: Regardless of the com- 
position of the population, strategy A has the higher payoff. Thus, evolutionary 
dynamics will always lead to a population with more A individuals than B indi- 
viduals. Equivalently, for a < c and b < d, strategy B dominates. In this case, 
the average abundance of B will be higher than the average abundance of A at the 
mutation-selection equilibrium. 

For a > c and b < d, both strategies are best replies to themselves. This is a 
'coordination game' . In a population of mostly A individual, rare B mutants have a 
lower payoff. In a population of mostly B individuals, rare A mutants have a lower 
payoff. Therefore, in a mutation selection process it is not a priori clear whether 
strategy A or strategy B will be more abundant at the equilibrium distribution. Two 
concepts are important: (i) pareto efficiency and (ii) risk-dominance. Strategy A is 
Pareto efficient if a > d. In this case, an aW-A population has a higher average 
payoff than an all--B popul ation. Strategy A is risk-dominant ifa + 6 > c + d 



(|Harsanyi and Seltenl . ll988|) . In this case, strategy A has a larger basin of attraction 
than strategy _B. In a game between two players, if I do not know what the other 
person will do, it is less risky for me to choose the risk dominant strategy, but it 
would be more rewarding for both of us to choose the Pareto efficient strategy. 
A coordination game is especially interesting, if the risk-dominant strategy is not 
Pareto efficient. 



(IKandori et al.l . 119931) have shown that A is chosen over B if a{N — 2) + bN > 
cN + d{N — 2). This means a large population selects the risk dominant equilib- 
rium in the long run. They analyze a process which is stochastic in the generation 
of mutants, but deterministic in following the gradient of selection. Their calcu- 
lation assumes asymptotically small mutation rates, but holds for a wide range of 
evolutionary processes, in contras t to previous approaches that make specific as 



sump tions on the source of noise ([Foster and YoungL Il99d ; iFudenberg and Harris . 



1992) . However, since the dynamics follows the gradient of selection, the processes 



can only leave a Nash equilibrium if a sufficient number of mutations occurs simul- 
taneously. In large populations, it is very unlikely that this happens, and the time 
until the population moves from one equilibrium to another becomes exponentially 



2 



large in A^. 



If the dyna mics is stochas tic, a single mutation can be sufficient to leave a Nash 
equilibrium. iNowak et al. have studied the competition of two strategies 

A and B in a frequency dependent Moran process (and similar processes) which 
model fully stochastic evolutionary dynamics. This means that the selection steps 
are stochastic. They have shown that the fixation probability of A is greater than 
that of B if and only if a{N - 2) + bN > cN + d{N - 2) . This calculation assumes 
weak selection in a process with selection only. The result is also valid for stronger 
inten s ities of selection, if w e use the pairwise comparison process (iSzabo and Toke , 
1998l : lTraulsen et al.l . l2007|) or a slightly mod ified version of the Moran process for 
evolutionary updating (ITraulsen et al.L 120081) . 



Here, we extend these results to both arbitrary mutation rates and arbitrary intensi- 
ties of selection. We show that A is more abundant than B in the mutation- selection 
distribution for a wide range of stochastic processes for any mutation rate and any 
intensity of selection if and only if a{N - 2) + bN > cN + d{N - 2). Higher 
mutation rates seem to be very relevant for social evolution, because humans (and 
higher animals) tend to try new strategies fairly often. We therefore expect that 
evolutionary dynamics in the cultural learning and imitation context operate under 
fairly high mutation (= 'exploration') rates. We also show that the condition holds 
for any payoff matrix, not only for coordination games. 



2 Pairwise comparison process 



The payoffs of A and B in a population of j individuals of type A and N — j 
individuals of type B are 

TT/i ? = a H 

N -I N -I 

Each individual interacts with A^— 1 other individuals. The payoff difference A7r(j) 
is a linear function of j 

^ , .s , , a — b — c + d. —a + bN — dN + d 
AvrO) = itaO) - nsij) = J + • (3) 



First, we need to specify how strategies spread in the population, depending on their 
payoffs. We adopt the following process: a random (focal) individual i is selected. It 
compares its payoff vr^ to the payoff vr^ of a randomly chosen role model, individual 

j, and takes over the strategy of that individual with probability 1 -i- 
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(lBlumeLll993l : ISzab6 and TokeLll998l : lTraulsen et alll2007l : ISandholmLl2007h . This 
process occurs with probability 1 — yU. With probability fi, a mutation occurs and the 
focal individual produces an offspring with random strategy, Aox B. The quantity 
(3 determines the intensity of selection (0 < /3 < oo). Strong selection (large (3) 
means that the probability to adopt a better strategy approaches one, and the proba- 
bility to adopt an inferior strategy vanishes. For weak selection (small (3), these two 
probabilities are close to 1/2. The transition probabilities to increase or decrease 
the number of A players by one, are given by 



3 N-3 



T; 



N N 1 + e-/5^^(j') 
J N-j 
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N 



J P 
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N 1 + e+/5^^(j) ^ iV 2 ■ 



(4) 



With these transition probabilities, we can determine whether A is more abundant 
than B or vice versa. Before discussing the problem for general mutation rates 
we first consider th e special case of low mutation rates , because of its importance 



in previous papers (IKandori et al.l . 1 19931 : iNowak et al.L l2004l : iTaylor et al.L 12004 



Imhof and Fudenberg . l2006h . We will also present a simple approximation for very 



high mutation rates, which illustrates the dynamics in this extreme limit. 



2. 1 Low mutation rates 



First, we consider low mutation rates, jj, <^ N^"^. In this limit, a mutation reaches 
extinction or fixation before a second mutation arises. Thus, the important quan- 
tity is the probabil ity that a single mu t ant takes over the population in a process 
without mutations ( Nowak et al.j l2004l : iTaylor et all l2004l : llmhof and Fudenberg , 
2006; Antal and Scheuring , .2006 1). The ratio of the fixation pro bability /?4 of a sin- 
gle A player and th at of a single B player can be written as (|Karlin and Tayloi . 
19751 : lNowadl2006h 



N-l 



T; 



PA 



(5) 



Without mutations, u = 0, the ratio of the transi tion probabilities @), simplifies to 
Tr/T+ = e-^^^(^) dTraulsen et al.l 



2006Ll2007h . Thus, Pb/pa reduces to 



PB 

— = exp 

PA 



N-l 



exp 



-/3 



' a + b 



d 



N -a + d 



(6) 



Therefore, = is equivalent to 



{a + b-c- d)N = 2a-2d. 



(7) 
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From the payoff matrix ^ it is clear, that strategy A is favored whenever a or 6 
increases, or c or decreases. Therefore, A is more abundant than B if 



a{N -2) + bN>cN + d{N - 2) 



(8) 



This condition has been derived previously in the low mutation limit for d ifferent 
evolutionary processes ( Kandori et all 119931 : iNowald 120061 : iTraulsen et al. 1. 12008,) . 
The strategy with the higher fixation probability is the more abundant one, because 
more of its mutants reach fixation. Thus for low mutation rates, A has a higher 
abundance than B whenever condition ([8]) is fulfilled. 

For weak selection, a wide range of processes fulfills the 1/3-rule, which states 
that the fixation probability of a single A mutant in a coordination game is larger 
than 1 /AT if a(iV- 2) + b(2N - 1) > c(N + 1) + d(2N - A) dNowak et allbooi 
Imhof and Nowakll2006l : lLessard and Ladretil2007l:lBomze and Pawlowitschll2008|) 



The fixation probability for large can thus be written as 



PA 




(9) 



where is a small positive number that may depend on the payoffs and on A^. 
Writing down the analogous equation for pB and subtracting yields 



Pa - Pb 



a — c 



d-h 



d—c—b+a a—b—c+d^ 



(X a + b 



d. 



(10) 



Thus, Pa > Pb is equivalent to a + b > c + d, which is identical to our condition 
dl]) for large A^. iLessard and Ladr et ('2007jLhave shown that any process within the 
domain of Kingman's coalescent (iKingmanlll982 ) fulfills the 1/3-rule (for large A^ 
and weak selection and mutation). If an evolutionary process fulfills the 1/3-rule, 
Pa and pb can be written in the form of dH). Hence we find again that Pa > Pb 
is equivalent to a + b > c + d. Thus, we conclude that our result is valid for any 
process within the domain of Kingman's coalescent for large A^ and weak selection 
and mutation. 



2.2 High mutation rates 



For high mutation rates p ^ 1, mutations dominate the process, and drive the 
system towards equal abundance of A and B, that is close to j = N/2. Hence the 
relevant transition probabilities are T^^^ ^i^d 1^ 1^ plausible to assume that 

T^^2 > implies that A is more abundant than B in the stationary state. From 
dH) we obtain 

- = ^ tanh (^^A7r{N/2)j , (11) 
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where the payoff difference is given by 



A.(iV/2) = i- + b-c-d)N-2ia-d) 

^ ' ' 2{N-l) ^ ^ 

Since /\tx{N/2) > implies T^^g ~ > 0' ^'^ ^^^^ implies a more abundant 
A, for arbitrary intensity of selection p. Thus, we have again the same condition 
a{N -2) + bN > cN + d{N - 2) for the dominance of A. 



2.3 Arbitrary mutation rates 



So far we have shown that both for low and high mutation rates the same condi- 
tion determines whether A or 5 is more abundant, regardless of the intensity of 
selection. Now, we turn to general mutation rates. Whenever equation ([7]) holds, 
the payoff difference Q has the following symmetry property 

A7r(j) = -A7r(iV-j). (13) 

This antisymmetry in the payoff differences then implies the following symmetry 
in our transition probabilities (S]), 

= T^-, . (14) 

For symmetric transition probabilities, however, the stationary probabilities pj of 
having j number of A players, are also symmetric, pj = Pn-j- Th is can be shown 



by first writing the stationar y probability distribution explicitly (IKampenl 119971 : 



Claussen and Traulsenl l2005b 



where po follows from the normalization J^jLoPj = 1- For symmetric transition 
probabilities (fT4l) and for j < N — j we can write 

Pj i=j -'i+l i=j -'■N-i-1 

Hence the distribution is symmetric pj = pn-j- 

For such a symmetric distribution, the number of A and B players are of course the 
same, since 

N N N 

(j) = E w = = - j)p^ = {N- j) . (17) 

i=o i=o j=o 
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Hence we have demonstrated that when d?]) is fulfilled, then the average abundance 
of A players and B players are the same. 

On the other hand, when condition ([8]) is fulfilled, then the average abundance of 
A players is higher than the average abundance of B players. This is quite obvious 
by looking at the payoff matrix ([I]): increasing a or b favors A, while increasing 
c or d favors B. More formally, we could say that from (|2l), © and @ it is clear 
that Tj~ and —Tj' are monotone increasing functions of a and b, and decreasing 
functions of c and d. This implies condition ([8]). Interestingly, other features of the 
game do not matter, e.g. whether a single A player has a larger disadvantage ma B 
population than a single B player in an A population. 

Condition ([8]) can also be written in the form 

a(l-|)+6>c + d(l-|), (18) 

which highlights the fact that there is a correction compared to simple risk 
dominance (a + b > c + d), which is the N ^ oo limit. This implies that a strat- 
egy can be more abundant than the other strategy for large A^, but less abundant 
for small N. For example for the payoff matrix ( ii f ) ^ strategy A is more abun- 
dant in large populations (N > 19), but B is more abundant in small populations 
(A^ < 17). They are equally abundant for = 18. In general, there is a threshold 
population size A^* = 2{a — d) /{a — d + b — c). If N* > 2, then A is more abundant 
than B either for A^ > A^* or for A^ < A^*. 



2.4 Including self-interactions 



So far, we have adopted the usual convention that individuals cannot interact with 
themselves. If instead we allow individuals to derive a payoff from self-interaction, 
we obtain 

j N-j ^ ^ 

fn(?') = — cH d. 

Hence, the payoff difference is A7r(j) = (a — 6 — c + d)j/N + b — d. Now 
for all mutation rates and all intensities of selection the condition for the average 
abundance of A to exceed the average abundance of B is simple risk dominance: 

a + b> c + d. (20) 



We conclude that the finite A^ correction to risk dominance in (1181) results from the 
exclusion of self-interactions. 
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3 Frequency dependent Moran process 



Although condition ([8]) is valid for a large class of evolutionary processes (see 
Sec.lH), it does not always hold if we depart from weak selection. As an example we 



disciiss the frequency dependent Moran process (INowak et all l2004l : iTaylor et al. . 



2004l : lAntal and ScheuringL 120061) . As we shall see, for this model our condition ([8]) 



is only valid in the limit of weak selection. 

Let the fitness be a convex combination of a background fitness (which we set to 1) 
and the payoff, /aO') = 1 — w + wtiaH) and fB{j) = l—w + w-Ksij). Here, w 
is the intensity of selection (0 < w < 1 for payoff matrices with positive entries). 
An individual is selected for reproduction at random, but proportional to fitness. 
The selected individual produces an offspring, which replaces a randomly chosen 
individual. Mutation can occur during reproduction. This leads to the transition 
probabilities 



^+_ jfAU)N-j ^^ , , iN-j)fBi3)N-j 
^ ~ Fij) N F{j) N ^ 

rr- _ {N -j)fB{j) J ,^ ^ , J J 



(21) 



Here F(j) = jfA{j) + {N — j)fB{j) is the total fitness of the whole population. 
These transition probabilities depend on the fitness values only through the ratio 
IaU) I /bO)- Yet, there is no simple condition for the equilibrium abundance of A 
players. In the weak selection limit, w ^ 1, however, only the payoff differences 
enter into the transition probabilities (|2T1) . 



Mzl = i + ^Avr(j) + 0(t/^'). (22) 

Thus, for weak selection, condition ([8]) again ensures the higher abundance of A. 

We can also consider a variant of the frequency dependent Moran process where 
fitness is an exponential function of payoff, /^^(j) = exp[+w7ryi(j)] and /^(j) = 
exp[+w7rB(j)] dTraulsen et al.ll2008[) . In this case, the transition probabilities again 
only depend on the payoff differences, because /aOO/ZbO) = exp(iyA7r(j)). 
Hence, the condition for abundance is again ([8]) for any intensity of selection and 
any mutation rate. 



4 General birth death processes 



Here, we show that our finding holds for a wide range of birth-death processes. We 
need two requirements to be fulfilled. The first is that the payoffs received from 
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other players should be additive. In this case the payoffs vr^O), t^bU), and also the 
payoff difference A7r(j) are linear functions of j. Consequently, 

A7r(j) + A7r(iV - j) = A(a, 6, c, rf, N) (23) 

is independent of j. Hence, when A (a, 6, c, d^N) = holds, we have an antisym- 
metric payoff function 

A7r(j) = -A7r(iV-j). (24) 

The second requirement is that the difference between the two types of players 
should manifest itself in the transition probabilities and T~ , only through the 
the payoff difference A7r(j). Hence, we can write Tj' = Tj^[A7r(j), /i], and the 
probability of decreasing the number of mutants is 

Tr = T,-[A7r(j),/i] = T+_^.[-A7r(j), /x]. (25) 

That is, the only difference between the two types is the change of sign of the payoff 
difference A7r(j). Otherwise, the transition probabilities are the same, but with the 
number of the opposite type of individuals N — j. Now, simply setting the index to 
— j in (|25l) , we have 

Tn-j = T;[-AniN-j),f^] = T,+ [A7r(j),/x] = T^. (26) 

For the second equality we have used (l24l) . These symmetric transition probabili- 
ties then imply an equal abundance of the two types of players in any birth-death 
process, as it has been shown in Section [231 The condition for a higher abundance 
of A compared to B is then A (a, b, c, d, N) > 0. This general condition takes the 
form of dH]) or (l20l) in our examples when self-intera ction are exclud ed or included. 



respectively. Note that similar arguments appear in (IClaussenl |2007|) . 



5 Discussion 



In this paper, we have shown that a{N — 2) + bN > cN + d{N — 2) is the crucial 
condition for A to be favored over B in wide range of evolutionary processes for 
any mutation rate, any intensity of selection and any finite population size. By A 
being favored over B we mean that A is more abundant than B in the mutation- 
selection equilibrium of the stochastic process. 

The relative abundance in the stationary state is a natural way of comparing two 
strategies. In the low mutation limit, when the population is typically homogeneous, 
a strategy being more abundant is equivalent of having a larger fixation probability. 
Hence, our study of abundance can be viewed as a generalization of that concept 
to arbitrary mutation rates. This is of special interest for cultural dynamics, where 
mutation rates are not necessarily small. 
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In particular, we have generalized the famous result of iKandori et al.1 (Il993h to 



any mutation rate. We have generalized the result of iNowak et al.l (|2004|) to any 
intensity of selection. Our results are valid for any mutation rate and any intensity 
of selection. 

It turns out that for our result to hold, a birth-death process has to fulfill two require- 
ments: (i) additive payoffs, and (ii) that that the evolutionary dynamics depend only 
on the payoff differences (the players are identical otherwise). These requirements 
hold, for example, for the pairwise comparison process described by Traulsen et 
al. (2006) and for a frequency dependent Moran process with exponential fitness 
function (Traulsen et al, 2008). For the standard frequency dependent Moran pro- 
cess with linear fitness function these requirements only hold in the limit of weak 
selection. 

Finally, we note that in coordination games with strong selection both the fixation 
probability of A and the fixation probability of B are very small. For small mutation 
rates, the system will stay in one equilibrium for a very long time. Therefore, it 
can take a very long time for the system to obtain a representative sample of the 
stationary mutation- selection distribution. 
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